Proximity triggered sampling

ABSTRACT

In one embodiment, a computer-implemented method comprising receiving data corresponding to an interaction with a user; based on the received data, predicting a moment in time when a state of the user is likely to change; and causing a change in one or a combination of message function characteristics or data collection function characteristics at the moment in time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims the priority benefit under 35 U.S.C. §119(e) of U.S. Provisional Application No. 62/527,087 filed on Jun. 30,2017, the contents of which are herein incorporated by reference.

FIELD OF THE INVENTION

The present invention is generally related to digital devices andsystems, and in particular, messaging using digital devices.

BACKGROUND OF THE INVENTION

Healthcare is beginning a dramatic transformation as the technologies ofthe Information Age are starting to be implemented to improve outcomesand lower costs. As part of that transformation, healthcare is becomingmore industrialized. More data than ever before is being collected aboutthe health and wellness of people, and this data is being used to drivebetter diagnoses and results based on what works across largepopulations. Best practices are being shared across hospitals andgeographic regions. At the same time, healthcare is becoming morepersonalized. For example, therapies are being tailored based on anindividual's genetics to make sure the best treatment for thatindividual is used. Finally, patients and caregivers are approachingtheir healthcare as consumers. In the healthcare system, they want theease, convenience, and simplicity that they already experience in theonline retail and financial space.

Despite those drivers of change in the healthcare space, it can be verychallenging to gather the data that actually furthers those aims,especially in the case of getting data related to a person's lifestyle.An experience sampling method (ESM), also referred to as a daily diarymethod, or ecological momentary assessment (EMA), is a researchmethodology that asks participants to stop at certain times and makenotes of their experience in real time. It is desirable to find theright moments to conduct ecological momentary assessments (EMA) orinterventions (EMI) (or in general to make certain functionalitiesavailable) to the users of a mobile device. Interventions may be in theform of electronic messages, and such timely messages may be a one-waycommunication message (e.g. information cards), without requiring theuser to provide any direct input, two-way communication messages (e.g.,mood rating questionnaires) where a user response is expected, orrequiring user input to activate certain functionality (e.g., start orstop heart rate (HR) measurements) in a device. A current method forfinding the right intervention moments is based on sampling atpre-determined or random moments and monitoring the user behavior andresponse rate with the help of the data mining tools. Such methodsprovide moments that are optimal across a population, but are likely tobe sub-optimal on an individual level. Some examples include the use ofa card-based system for interacting with users of the PhilipsHealthwatch, where messages are presented as cards. In most cases thesecards are one-way messages (e.g., informational), but in some cases acard poses a question to the user. The timing of the message may varydepending on the difference between one that is utilized by the user inan impactful way and one that is ignored.

SUMMARY OF THE INVENTION

In one embodiment, a computer-implemented method comprising receivingdata corresponding to an interaction with a user; based on the receiveddata, predicting a moment in time when a state of the user is likely tochange; and causing a change in one or a combination of message functioncharacteristics or data collection function characteristics at themoment in time.

These and other aspects of the invention will be apparent from andelucidated with reference to the embodiment(s) described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the invention can be better understood with reference tothe following drawings, which are diagrammatic. The components in thedrawings are not necessarily to scale, emphasis instead being placedupon clearly illustrating the principles of the present invention.Moreover, in the drawings, like reference numerals designatecorresponding parts throughout the several views.

FIG. 1 is a schematic diagram that illustrates an example environment inwhich a proximity triggered sampling system is used, in accordance withan embodiment of the invention.

FIG. 2 is a schematic diagram that illustrates an example wearabledevice in which all or a portion of the functionality of a proximitytriggered sampling system may be implemented, in accordance with anembodiment of the invention.

FIG. 3 is a schematic diagram that illustrates an example electronicsdevice in which all or a portion of the functionality of a proximitytriggered sampling system may be implemented, in accordance with anembodiment of the invention.

FIG. 4 is a schematic diagram that illustrates an example computingdevice in which all or a portion of the functionality of a proximitytriggered sampling system may be implemented, in accordance with anembodiment of the invention.

FIG. 5 is a schematic diagram that illustrates example changes inproximity information as a function of time, in accordance with anembodiment of the invention.

FIG. 6 is a flow diagram that illustrates an example proximity triggeredsampling process, in accordance with an embodiment of the invention.

FIG. 7 is a schematic diagram that illustrates an example proximitygraph, in accordance with an embodiment of the invention.

FIG. 8 is a flow diagram that illustrates an example proximity triggeredsampling method, in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Disclosed herein are certain embodiments of a proximity triggeredsampling system, apparatus, and method (also collectively referred toherein as a proximity triggered sampling system) that determine the bestmoments to intervene in a user's life with messaging via a user device.In one embodiment, the proximity triggered sampling system determinesthe best moments by finding meaningful sampling moments via user statemonitoring using data (e.g., proximity data). In other words, theproximity triggered sampling system samples the psychological (and/orphysiological) state of a person in relation to social interactions,from there determining the best moments to intervene with questions,content, and/or to change device functionality characteristics. Forinstance, the proximity triggered sampling system may monitor if theperson is alone, or together with someone, to determine the moments tosend mood questionnaires.

Digressing briefly, existing systems start sampling at random (orcalculated) times to find the right moments to intervene, and once asufficient amount of data has been collected, to use this data topredict/calculate new times, which is not an optimal method. Forinstance, random sampling requires many observations, which means alarge amount of user sampling to find the best intervention moments.Since the sampling moments are selected randomly, it is highly likelythat the user state has not changed, which limits the information thatcan be learned from the sampling. Certain embodiments of a proximitytriggered sampling system, in contrast, predict the moments where userstate (e.g. psychological and/or physiological) is likely to change, andincrease the sampling frequency at these meaningful moments, or sampleonly at these moments. For example, for an ecological momentaryassessment (EMA) that includes a mood monitoring questionnaire,predicting the moments of potential user emotional state change, andusing these moments to trigger user sampling (e.g., an EMA) is a betterand more effective method. One challenge to this approach is (to learn)to automatically predict the meaningful moments, where the user'semotional state is likely to change. With certain embodiments of aproximity triggered sampling system, one solution, supported bypreliminary experimental data, is based on observing the user with, insome instances, the same device that delivers the EMAs (hence not alwaysrequiring additional sensors) to find better moments for sampling theuser.

Having summarized certain features of a proximity triggered samplingsystem of the present disclosure, reference will now be made in detailto the description of a proximity triggered sampling system asillustrated in the drawings. While a proximity triggered sampling systemwill be described in connection with these drawings, there is no intentto limit the proximity triggered sampling system to the embodiment orembodiments disclosed herein. For instance, though described primarilyin the context of directly soliciting user input using questions (e.g.,two-way communication messages), certain embodiments of a proximitytriggered sampling system may be used to push information to the user atthe meaningful moments (e.g., a one-way communication message, includinginformation cards, that do not require the user to provide input) and/orchange (e.g., activate, increase, decrease) data collectionfunctionality characteristics (e.g., start or stop a heart ratemeasurement, sampling frequency, memory capacity, etc.) of the maincommunication device or other accompanying devices or software linked tothe main communication device. Further, although the descriptionidentifies or describes specifics of one or more embodiments, suchspecifics are not necessarily part of every embodiment, nor are allvarious stated advantages necessarily associated with a singleembodiment or all embodiments. On the contrary, the intent is to coverall alternatives, modifications and equivalents consistent with thedisclosure as defined by the appended claims. Further, it should beappreciated in the context of the present disclosure that the claims arenot necessarily limited to the particular embodiments set out in thedescription.

Referring now to FIG. 1, shown is an example environment 10 in whichcertain embodiments of a proximity triggered sampling system may beimplemented. It should be appreciated by one having ordinary skill inthe art in the context of the present disclosure that the environment 10is one example among many, and that some embodiments of a proximitytriggered sampling system may be used in environments with fewer,greater, and/or different components that those depicted in FIG. 1. Theenvironment 10 comprises a plurality of devices that enablecommunication of information throughout one or more networks. Thedepicted environment 10 comprises a wearable device 12, an electronicsdevice 14, a cellular network 16, a wide area network 18 (e.g., alsodescribed herein as the Internet), and a remote computing system 20comprising one or more computing devices and/or storage devices. Notethat the wearable device 12 and the electronics device 14 are alsoreferred to as user devices. The wearable device 12, as describedfurther in association with FIG. 2, is typically worn by the user (e.g.,around the wrist or torso or attached to an article of clothing), andcomprises a plurality of sensors that track physical activity of theuser (e.g., steps, swim strokes, pedaling strokes, sports activities,etc.), sense/measure or derive physiological parameters (e.g., heartrate, respiration, skin temperature, etc.) based on the sensor data, andoptionally sense various other parameters (e.g., outdoor temperature,humidity, location, etc.) pertaining to the surrounding environment ofthe wearable device 12. For instance, in some embodiments, the wearabledevice 12 may comprise a global navigation satellite system (GNSS)receiver (and associated positioning software and antenna(s)), includinga GPS receiver, which tracks and provides location coordinates (e.g.,latitude, longitude, altitude) for the device 12. Other informationassociated with the recording of coordinates may include speed,accuracy, and a time stamp for each recorded location. In someembodiments, the location information may be in descriptive form, andgeofencing (e.g., performed locally or external to the wearable device12) is used to transform the descriptive information into coordinatenumbers. In some embodiments, the wearable device 12 may comprise indoorlocation or proximity sensing technology, including beacons, RFID orother coded light technologies, Wi-Fi, etc. In some embodiments, GNSSfunctionality may be performed at the electronics device 14 in additionto, or in lieu of, such functionality being performed at the wearabledevice 12. Some embodiments of the wearable device 12 may include amotion or inertial tracking sensor, including an accelerometer and/or agyroscope, providing movement data of the user (e.g., to detect limbmovement and type of limb movement to facilitate the determination ofwhether the user is engaged in sports activities, stair walking, orbicycling, or the provision of other contextual data). A representationof such gathered data may be communicated to the user via an integrateddisplay on the wearable device 12 and/or on another device or devices.In some embodiments, the wearable device 12 may be embodied as a virtualreality device or an augmented reality device. In some embodiments, thewearable device 12 may be embodied as an implantable, which may includebiocompatible sensors that reside underneath the skin or are implantedelsewhere.

Also, such data gathered by the wearable device 12 may be communicated(e.g., continually, periodically, and/or aperiodically, including uponrequest) to one or more electronics devices, such as the electronicsdevice 14 or to the computing system 20 via the cellular network 16.Such communication may be achieved wirelessly (e.g., using near fieldcommunications (NFC) functionality, Blue-tooth functionality,802.11-based technology, etc.) and/or according to a wired medium (e.g.,universal serial bus (USB), etc.). Further discussion of the wearabledevice 12 is described below in association with FIG. 2.

The electronics device 14 may be embodied as a smartphone, mobile phone,cellular phone, pager, stand-alone image capture device (e.g., camera),laptop, workstation, smart glass (e.g., Google Glass™), virtual realitydevice, augmented reality device, smart watch, among other handheld andportable computing/communication devices. In some embodiments, theelectronics device 14 is not necessarily readily portable or evenportable. For instance, the electronics device 14 may be a homeappliance, including a refrigerator, microwave, oven, pillbox, homemonitor, stand-alone home virtual assistant device, one or more of whichmay be coupled to the computing system 20 via one or more networks(e.g., through the home Internet connection or telephony network), or avehicle appliance (e.g., the automobile navigation system orcommunication system). In the depicted embodiment of FIG. 1, theelectronics device 14 is a smartphone, though it should be appreciatedthat the electronics device 14 may take the form of other types ofdevices including those described above. Further discussion of theelectronics device 14 is described below in association with FIG. 3,with smartphone and electronics device 14 used interchangeablyhereinafter.

In one embodiment, the wearable device 12 or electronics device 14individually comprise the full functionality of the proximity triggeredsampling system. In some embodiments, the wearable device 12 andelectronics device 14 collectively comprise the full functionality ofthe proximity triggered sampling system (e.g., the functionality of theproximity triggered sampling system is distributed among the twodevices). In some embodiments, functionality of the proximity triggeredsampling system is distributed among the wearable device 12 (orelectronics device 14) and the computing system 20. In some embodiments,functionality of the proximity triggered sampling system is distributedamong the wearable device 12, the electronics device 14, and thecomputing system 20. For instance, the wearable device 12 and/or theelectronics device 14 may present electronic messages via a userinterface and provide sensing functionality, yet rely on the remotecomputing systems 20 for processing and/or data (e.g., message) storage.

The cellular network 16 may include the necessary infrastructure toenable cellular communications by the electronics device 14 andoptionally the wearable device 12. There are a number of differentdigital cellular technologies suitable for use in the cellular network16, including: GSM, GPRS, CDMAOne, CDMA2000, Evolution-Data Optimized(EV-D0), EDGE, Universal Mobile Telecommunications System (UMTS),Digital Enhanced Cordless Telecommunications (DECT), Digital AMPS(IS-136/TDMA), and Integrated Digital Enhanced Network (iDEN), amongothers.

The wide area network 18 may comprise one or a plurality of networksthat in whole or in part comprise the Internet. The electronics device14 and optionally wearable device 12 may access one or more devices ofthe computing system 20 via the Internet 18, which may be furtherenabled through access to one or more networks including PSTN (PublicSwitched Telephone Networks), POTS, Integrated Services Digital Network(ISDN), Ethernet, Fiber, DSL/ADSL, among others.

The computing system 20 comprises one or more devices coupled to thewide area network 18, including one or more computing devices networkedtogether, including an application server(s) and data storage. Thecomputing system 20 may serve as a cloud computing environment (or otherserver network) for the electronics device 14 and/or wearable device 12,performing processing and data storage on behalf of (or in someembodiments, in addition to) the electronics devices 14 and/or wearabledevice 12. When embodied as a cloud service or services, the device(s)of the remote computing system 20 may comprise an internal cloud, anexternal cloud, a private cloud, or a public cloud (e.g., commercialcloud). For instance, a private cloud may be implemented using a varietyof cloud systems including, for example, Eucalyptus Systems, VMWarevSphere®, or Microsoft® HyperV. A public cloud may include, for example,Amazon EC2®, Amazon Web Services®, Terremark®, Savvis®, or GoGrid®.Cloud-computing resources provided by these clouds may include, forexample, storage resources (e.g., Storage Area Network (SAN), NetworkFile System (NFS), and Amazon S3®), network resources (e.g., firewall,load-balancer, and proxy server), internal private resources, externalprivate resources, secure public resources, infrastructure-as-a-services(IaaSs), platform-as-a-services (PaaSs), or software-as-a-services(SaaSs). The cloud architecture of the devices of the remote computingsystem 20 may be embodied according to one of a plurality of differentconfigurations. For instance, if configured according to MICROSOFTAZURE™, roles are provided, which are discrete scalable components builtwith managed code. Worker roles are for generalized development, and mayperform background processing for a web role. Web roles provide a webserver and listen for and respond to web requests via an HTTP (hypertexttransfer protocol) or HTTPS (HTTP secure) endpoint. VM roles areinstantiated according to tenant defined configurations (e.g.,resources, guest operating system). Operating system and VM updates aremanaged by the cloud. A web role and a worker role run in a VM role,which is a virtual machine under the control of the tenant. Storage andSQL services are available to be used by the roles. As with otherclouds, the hardware and software environment or platform, includingscaling, load balancing, etc., are handled by the cloud.

In some embodiments, the devices of the remote computing system 20 maybe configured into multiple, logically-grouped servers (run on serverdevices), referred to as a server farm. The devices of the remotecomputing system 20 may be geographically dispersed, administered as asingle entity, or distributed among a plurality of server farms,executing one or more applications on behalf of one or more of theelectronic devices 14 and/or wearable device 12. The devices of theremote computing system 20 within each farm may be heterogeneous. One ormore of the devices may operate according to one type of operatingsystem platform (e.g., WIND0WS NT, manufactured by Microsoft Corp. ofRedmond, Wash.), while one or more of the other devices may operateaccording to another type of operating system platform (e.g., Unix orLinux). The group of devices of the remote computing system 20 may belogically grouped as a farm that may be interconnected using a wide-areanetwork (WAN) connection or medium-area network (MAN) connection. Thedevices of the remote computing system 20 may each be referred to as(and operate according to) a file server device, application serverdevice, web server device, proxy server device, or gateway serverdevice.

In one embodiment, the computing system 20 may comprise a web serverthat provides a web site that can be used by users to receive messagesand/or input responses to a questionnaire, though other platforms may beused to enable user input or the receipt of electronic messages,including functionality that provides push notifications or device-runlocal applications. The computing system 20 may receive information(e.g., proximity data, physiological data, etc.) collected via one ormore of the wearable device 12 or electronics device 14 and/or otherdevices or applications, store the received information in a userprofile data structure (e.g., database) and/or other memory, process theinformation to determine a user state (e.g., psychological and/orphysiological state of a user based in part on the proximity data andphysiological data), and deliver electronic messages and/or solicitinformation from the user (e.g., mood questionnaire, etc.) via theelectronics device 14 and/or wearable device 12. In some embodiments,the computing system 20 may change a data collection functioncharacteristic(s), such as via sending instructions to the devices 12and/or 14 to start/end measurements (e.g., heart rate measurements),track location (e.g., using GNSS functionality), change sampling rates,etc. Note that in some embodiments, all or at least a portion of theaforementioned computing system functionality may be implemented at thewearable device 12 and/or electronics device 14. The computing system 20is programmed to handle the operations of one or more applications(e.g., health or wellness programs) implemented also on the wearabledevice 12 and/or electronics device 14 via the networks 16 and/or 18.For example, the computing system 20 processes user registrationrequests, user device activation requests, user information updatingrequests, data uploading requests, data synchronization requests, etc.The data received at the computing system 20 may include a plurality ofmeasurements pertaining to sensed or determined parameters, for example,proximity data, body movements and activities, heart rate, respirationrate, blood pressure, body temperature, light and visual information,etc. and the corresponding context. Such measurements and/or informationderived from the measurements may provide insight into the user state(e.g., the emotional, physical, or psychological state of the user),enabling the determination of opportune (meaningful) moments for changein message function characteristics and/or data collection functioncharacteristics (e.g., change being activation, increase or decrease insampling, etc.), which includes delivery of the messages and/oractivation and/or change in data collection functionality. In someembodiments, the computing system 20 is configured to be a backendserver for a health-related program or a health-related applicationimplemented on the mobile devices. The functions of the computing system20 described above are for illustrative purpose only. The presentdisclosure is not intended to be limiting. The computing system 20 mayinclude one or more general computing server devices or dedicatedcomputing server devices. The computing system 20 may be configured toprovide backend support for a program developed by a specificmanufacturer. However, the computing system 20 may also be configured tobe interoperable across other server devices and generate information ina format that is compatible with other programs. In some embodiments,one or more of the functionality of the computing system 20 may beperformed at the respective devices 12 and/or 14. Further discussion ofthe computing system 20 is described below in association with FIG. 4.

As one illustrative example of operations of an embodiment of aproximity triggered sampling system, in accordance with a client-serverapplication (e.g., fitness or health related application, though othertypes of applications including those for business, rehab, training,instruction, etc. may be run), the wearable device 12 and/or electronicsdevice 14 may be equipped with proximity sensing functionality (e.g.,beacon technology) along with other sensing functionality (e.g.,accelerometer, position detection, physiological monitoring, etc.).Social interactions between the user of the wearable device 12 and/orelectronics device 14 and the users of other devices are monitored usingthe proximity sensing functionality (and in some embodiments, usingother user data), including such social interaction characteristics asnumber of people involved in the interactions, the duration, location,characteristics of people involved, time (e.g., in a day) of theinteraction, the duration between different interactions, the count ofthe interaction (e.g., is this the first, second, etc. interaction thatthe user had), changes observed in the proximity data during theinteraction (e.g., people coming, people leaving), and in general,interactions between different interaction characteristics. Forinstance, the proximity triggered sampling system assesses interactionsinvolving two one to one meetings (1-1, 1-1) and then a meeting with 30people. It may be the case that due to the 1-1 meetings in the past, theuser may react differently to the later meetings. In other words,certain embodiments of a proximity triggered sampling system considerhistorical data in assessments of the social interactions.

The proximity data, which includes the social interactioncharacteristics, and optionally other sensor data, may continually becommunicated by the wearable device 12 and/or electronics device 14 tothe computing system 20. Software in the computing system 20 uses thesensor data to predict when a user state of the user of the wearabledevice 12 and/or electronics device 14 is likely to change, and thenchanges message function characteristics (e.g., communicates a one-wayor two-way message to the user, number and/or type of questions asked,etc.) and/or data collection function characteristics (e.g., control thebeginning and end of a heart rate measurement, position tracking, etc.).In some embodiments, the application software to control functionalityof the computing system 20 may be implemented entirely in the wearabledevice 12, the electronics device 14, and/or distributed among bothdevices 12 and 14 or all devices 12, 14, and device(s) of the computingsystem 20. As indicated above, the wearable device 12, electronic device14, and the computing system 20 may operate under a health and wellnessprogram that monitors the performance and/or progress of the user underthe program and provides electronic messaging to the user that has anoptimal impact on the user (e.g., via the pushing of appropriate one-waycommunication messages for the current context of the user, pushingtwo-way messages to ascertain an emotional state of the user, orcollecting data). For instance, the user input data can be minedtogether with the heart rate (HR) data, for determining later samplingmoments. In some embodiments, the collected data may be used tocalculate an indicator of the difference between subjective (e.g. userinput based emotions) and objective measurements (e.g. HR basedemotions) and these can be used to tailor one or more samplingcharacteristics. In some applications, including rehabilitation, it maybe desirable to gather high fidelity data that would enable calculatingof the additional parameters, such as heart rate variability (HRV). HRVrequires that the heart rate signal be of higher quality, withindividual cycle peaks being detectable, and for at least a certainduration (e.g., for a suggested duration equal to a minimum of 5minutes.) For people suffering from anxiety, stress or mood swings, itmay be very relevant to have access to HRV data. In cases where thereare physical limitations in the devices (e.g. limited storage, memory,battery), it is beneficial to activate and deactivate the HR only atcertain moments to conserve resources.

Attention is now directed to FIG. 2, which illustrates an examplewearable device 12 in which all or a portion of the functionality of aproximity triggered sampling system may be implemented. That is, FIG. 2illustrates an example architecture (e.g., hardware and software) forthe example wearable device 12. It should be appreciated by one havingordinary skill in the art in the context of the present disclosure thatthe architecture of the wearable device 12 depicted in FIG. 2 is but oneexample, and that in some embodiments, additional, fewer, and/ordifferent components may be used to achieve similar and/or additionalfunctionality. In one embodiment, the wearable device 12 comprises aplurality of sensors 22 (e.g., 22A-22N), one or more signal conditioningcircuits 24 (e.g., SIG COND CKT 24A-SIG COND CKT 24N) coupledrespectively to the sensors 22, and a processing circuit 26 (PROCES CKT)that receives the conditioned signals from the signal conditioningcircuits 24. In one embodiment, the processing circuit 26 comprises ananalog-to-digital converter (ADC), a digital-to-analog converter (DAC),a microcontroller unit (MCU), a digital signal processor (DSP), andmemory (MEM) 28. In some embodiments, the processing circuit 26 maycomprise fewer or additional components than those depicted in FIG. 2.For instance, in one embodiment, the processing circuit 26 may consistentirely of the microcontroller. In some embodiments, the processingcircuit 26 may include the signal conditioning circuits 24. The memory28 comprises an operating system (OS) and application software (ASW) 30,which may be used to execute a particular health and wellness programfor a user (or in some embodiments, business program, rehabilitationprogram, training program, etc.). In the depicted embodiment, theapplication software 30 comprises a sensor measurement module (SMM) 32,a prediction module (PM) 34, an interface module (IM) 36, and acommunications module (CM) 38. In some embodiments, additional modulesused to achieve the disclosed functionality of a proximity triggeredsampling system, among other functionality, may be included, or one ormore of the modules 32-38 may be separate from the application software30. In some embodiments, fewer than all of the modules 32-38 may be usedin the wearable device 12, such as in embodiments where the wearabledevice 12 merely provides sensor functionality for communication ofsensor data to one or more other devices.

The sensor measurement module 32 in cooperation with the sensors 22collectively comprise a data collection function for the wearable device12. In one embodiment, the sensor measurement module 32 comprisesexecutable code (instructions) to process the signals (and associateddata) measured by the sensors 22 and record and/or derive physiologicalparameters, such as heart rate, blood pressure, respiration,perspiration, etc. and movement and/or location data. In someembodiments, the sensors 22 comprise beacon technology (e.g., usingBluetooth Low Energy, such as found in iBeacon technology), where thebeacons provide proximity data for use by the prediction module 34(along with other data in some embodiments). In some embodiments, thesensor measurement module 32 may comprise location positioning software(e.g., in cooperation with GNSS receiver functionality included amongthe sensors 22). The measured information (or information derived fromthe measured information) may be used in some embodiments of a proximitytriggered sampling system to provide context about a user'spsychological and/or physiological state, as used by the predictionmodule 34 to predict a moment in time when a change state of a user islikely.

The prediction module 34, explained further below, receives data (e.g.,proximity data, physiological data, and optionally other contextualdata) and repeatedly generates proximity graphs in real time. Bymonitoring the proximity data, changes in social interactions with theuser may be determined, enabling the prediction module 34 to predictwhen the state of a user is likely to change and selecting those momentsto change communication function characteristics and/or data collectionfunction characteristics. One result of prediction can be to initiate acommunication with a user (e.g., present a message) and/or change a type(e.g., content) and/or frequency or timing of the message. The messagecan be presented as a one-way message (no user response required), or asa two-way message wherein an initiation of a dialog takes place wheresome user input/response is solicited or expected. Another result of theprediction may be to change data collection function characteristics ofa user device. For example, a sensor 22 in the wearable device 12 (e.g.for HR measurements) may be activated, or position detection trackingfunctionality (e.g., GNSS functionality) may be activated, and/or achange in the setting of the device 12 (e.g. increase of the samplingfrequency of accelerometer and HR sensor) may be realized. Anotherexample for a result of the prediction is the storage of parameters in adatabase. For instance, by storing the parameters, the parameters may beprocessed and used to improve the prediction algorithms. In other words,in some embodiments, the proximity triggered sampling system functionsas a dynamic system that learns with every new set of data. When it isdetected that the user's state may have changed, it is important to havethe data stored. In one embodiment, the proximity triggered samplingsystem may process and display the sensor data in real-time, but onlystart storing the data in cases and periods where the users state changehas been predicted, and the sampling characteristics adapted. In oneembodiment, the proximity triggered sampling system may store data onlywhen the user has responded to the initiated communication, enabling avaluable result that the data may be labelled. Having good, well-labeleddata is beneficial, and thus by storing data when the user hasresponded, the user's responses can be used as labels for the data(which later may be used in the training of classification algorithms).One or any combination of these results may be implemented in thewearable device 12 (or other devices 14 or device(s) of the computingsystem 20 in which the prediction module 34 is used). In someembodiments, the result of the prediction is that nothing further isdone (no change in operations).

The interface module 36 comprises executable code (instructions) toenable the presentation of electronic messages to the user at a momentin time that is meaningful (corresponding to the time when the userstate is likely to change). In some embodiments, the interface module 36may comprise functionality to enable the input (via voice or typed entryor gestures of different body parts such as hands, head, etc.) of userinput, including in response to questions presented in the electronicmessages. In one embodiment, the interface module 36 generates agraphical user interface (GUI) on a display screen of the wearabledevice 12, and/or provides other functionality, including in someembodiments virtual or augmented reality type functionality.

The communications module 38 comprises executable code (instructions) toenable a communications circuit 40 of the wearable device 12 to operateaccording to one or more of a plurality of different communicationtechnologies (e.g., NFC, Bluetooth, Zigbee, etc.). In some embodiments,the communications module 38 provides control for any beacons deliveredand/or detected by at least one of the sensors 22. In some embodiments,the communications module 38 may perform beacon transmission andreception, though still referred to herein as a sensing function. Forpurposes of illustration, the communications module 38 is describedherein as providing for control of communications with the electronicsdevice 14 and/or the computing system 20 (FIG. 1). In an embodimentwhere the prediction functionality is performed at the electronicsdevice 14 or the computing system 20, the communications module 38, incooperation with the communications circuit 40, may provide for thetransmission of raw sensor data and/or the derived information from thesensor data to the electronics device 14 for processing by theelectronics device 14, or to the computing system 20 (directly via thecellular network 16 and/or Internet or via the electronics device 14)for processing at the computing system 20. The communications module 38,in cooperation with the communications circuit 40, may receive anymessages (or instructions to activate and/or increase/decrease datacollection function characteristics) from the electronics device 14 orthe computing system 20 for presentation to the user at the wearabledevice 12. In some embodiments, the presentation of the messages mayoccur at the electronics device 14. In some embodiments, thecommunications module 38 may also include browser software in someembodiments to enable Internet connectivity, and may also be used toaccess certain services, such as mapping/place location services, whichmay be used to determine a context for the sensor data. These servicesmay be used in some embodiments of a proximity triggered samplingsystem, and in some instances, may not be used. In some embodiments, thelocation services may be performed by a client-server applicationrunning on the electronics device 14 and a device of the remotecomputing system 20. Further, some embodiments of the wearable device 12may comprise only a portion of the functionality of an embodiment of aproximity triggered sampling system.

As indicated above, in one embodiment, the processing circuit 26 iscoupled to the communications circuit 40. The communications circuit 40serves to enable wireless communications between the wearable device 12and other devices, including the electronics device 14 and/or in someembodiments, device(s) of the computing system 20, among other devices.The communications circuit 40 is depicted as a Bluetooth circuit, thoughnot limited to this transceiver configuration. For instance, in someembodiments, the communications circuit 40 may be embodied as any one ora combination of an NFC circuit, Wi-Fi circuit, transceiver circuitrybased on Zigbee, 802.11, GSM, LTE, CDMA, WCDMA, among others such asoptical or ultrasonic based technologies.

The processing circuit 26 is further coupled to input/output (I/O)devices or peripherals, including an input interface 42 (INPUT) and theoutput interface 44 (OUT). Note that in some embodiments, functionalityfor one or more of the aforementioned circuits and/or software may becombined into fewer components/modules, or in some embodiments, furtherdistributed among additional components/modules or devices. Forinstance, the processing circuit 26 may be packaged as an integratedcircuit that includes the microcontroller (microcontroller unit or MCU),the DSP, and memory 28, whereas the ADC and DAC may be packaged as aseparate integrated circuit coupled to the processing circuit 26. Insome embodiments, one or more of the functionality for the above-listedcomponents may be combined, such as functionality of the DSP performedby the microcontroller.

The sensors 22 are selected to perform detection and measurement of aplurality of physiological and behavioral parameters. For instance,typical physiological parameters include heart rate, heart ratevariability, heart rate recovery, blood flow rate, activity level,muscle activity (e.g., movement of limbs, repetitive movement, coremovement, body orientation/position, power, speed, acceleration, etc.),muscle tension, blood volume, blood pressure, blood oxygen saturation,respiratory rate, perspiration, skin temperature, electrodermal activity(skin conductance response), body weight, and body composition (e.g.,body mass index or BMI), articulator movements (especially duringspeech). Typical behavioral parameters or activities including walking,running, cycling, and/or other activities, including shopping, walking adog, working in the garden, sports activities, browsing internet,watching TV, typing, etc.). At least one of the sensors 22 may beembodied as movement and/or pressure detecting sensors, includinginertial sensors (e.g., gyroscopes, single or multi-axis accelerometers,such as those using piezoelectric, piezoresistive or capacitivetechnology in a microelectromechanical system (MEMS) infrastructure forsensing movement). In some embodiments, at least one of the sensors 22may include GNSS sensors, including a GPS receiver to facilitatedeterminations of distance, speed, acceleration, location, altitude,etc. (e.g., location data, or generally, sensing movement), in additionto or in lieu of the accelerometer/gyroscope and/or indoor tracking(e.g., Wi-Fi, coded-light based technology, etc.). In some embodiments,GNSS sensors (e.g., GNSS receiver and antenna(s)) may be included in theelectronics device 14 in addition to, or in lieu of, those residing inthe wearable device 12. The sensors 22 may also include flex and/orforce sensors (e.g., using variable resistance), electromyographicsensors, electrocardiographic sensors (e.g., EKG, ECG), magneticsensors, photoplethysmographic (PPG) sensors, bio-impedance sensors,infrared proximity sensors, acoustic/ultrasonic/audio sensors, a straingauge, galvanic skin/sweat sensors, pH sensors, temperature sensors,pressure sensors, and photocells. The sensors 22 may include otherand/or additional types of sensors for the detection of environmentalparameters and/or conditions, for instance, barometric pressure,humidity, outdoor temperature, pollution, noise level, etc. One or moreof these sensed environmental parameters/conditions may be influentialin the determination of the state of the user. In some embodiments, thesensors 22 include proximity sensors (e.g., iBeacon® and/or otherindoor/outdoor positioning functionality, including those based on Wi-Fior dedicated sensors), that are used to determine proximity of thewearable device 12 to other devices that also are equipped with beaconor proximity sensing technology. In some embodiments, GNSS functionalityand/or the beacon functionality may be achieved via the communicationscircuit 40 or other circuits coupled to the processing circuit 26.

The signal conditioning circuits 24 include amplifiers and filters,among other signal conditioning components, to condition the sensedsignals including data corresponding to the sensed physiologicalparameters and/or location signals before further processing isimplemented at the processing circuit 26. Though depicted in FIG. 2 asrespectively associated with each sensor 22, in some embodiments, fewersignal conditioning circuits 24 may be used (e.g., shared for more thanone sensor 22). In some embodiments, the signal conditioning circuits 24(or functionality thereof) may be incorporated elsewhere, such as in thecircuitry of the respective sensors 22 or in the processing circuit 26(or in components residing therein). Further, although described aboveas involving unidirectional signal flow (e.g., from the sensor 22 to thesignal conditioning circuit 24), in some embodiments, signal flow may bebi-directional. For instance, in the case of optical measurements, themicrocontroller may cause an optical signal to be emitted from a lightsource (e.g., light emitting diode(s) or LED(s)) in or coupled to thecircuitry of the sensor 22, with the sensor 22 (e.g., photocell)receiving the reflected/refracted signals. In the case of beacontechnology, beacons may be transmitted from and received by at least oneof the sensors 22.

The communications circuit 40 is managed and controlled by theprocessing circuit 26 (e.g., executing the communications module 38).The communications circuit 40 is used to wirelessly interface with theelectronics device 14 (FIG. 3) and/or in some embodiments, one or moredevices of the computing system 20. In one embodiment, thecommunications circuit 40 may be configured as a Bluetooth transceiver,though in some embodiments, other and/or additional technologies may beused, such as Wi-Fi, GSM, LTE, CDMA and its derivatives, Zigbee, NFC,among others. In the embodiment depicted in FIG. 2, the communicationscircuit 40 comprises a transmitter circuit (TX CKT), a switch (SW), anantenna, a receiver circuit (RX CKT), a mixing circuit (MIX), and afrequency hopping controller (HOP CTL). The transmitter circuit and thereceiver circuit comprise components suitable for providing respectivetransmission and reception of an RF signal, including amodulator/demodulator, filters, and amplifiers. In some embodiments,demodulation/modulation and/or filtering may be performed in part or inwhole by the DSP. The switch switches between receiving and transmittingmodes. The mixing circuit may be embodied as a frequency synthesizer andfrequency mixers, as controlled by the processing circuit 26. Thefrequency hopping controller controls the hopping frequency of atransmitted signal based on feedback from a modulator of the transmittercircuit. In some embodiments, functionality for the frequency hoppingcontroller may be implemented by the microcontroller or DSP. Control forthe communications circuit 40 may be implemented by the microcontroller,the DSP, or a combination of both. In some embodiments, thecommunications circuit 40 may have its own dedicated controller that issupervised and/or managed by the microcontroller.

In one example operation for the communications circuit 40, a signal(e.g., at 2.4 GHz) may be received at the antenna and directed by theswitch to the receiver circuit. The receiver circuit, in cooperationwith the mixing circuit, converts the received signal into anintermediate frequency (IF) signal under frequency hopping controlattributed by the frequency hopping controller and then to baseband forfurther processing by the ADC. On the transmitting side, the basebandsignal (e.g., from the DAC of the processing circuit 26) is converted toan IF signal and then RF by the transmitter circuit operating incooperation with the mixing circuit, with the RF signal passed throughthe switch and emitted from the antenna under frequency hopping controlprovided by the frequency hopping controller. The modulator anddemodulator of the transmitter and receiver circuits may performfrequency shift keying (FSK) type modulation/demodulation, though notlimited to this type of modulation/demodulation, which enables theconversion between IF and baseband. In some embodiments,demodulation/modulation and/or filtering may be performed in part or inwhole by the DSP. The memory 28 stores the communications module 38,which when executed by the microcontroller, controls the Bluetooth(and/or other protocols) transmission/reception.

Though the communications circuit 40 is depicted as an IF-typetransceiver, in some embodiments, a direct conversion architecture maybe implemented. As noted above, the communications circuit 40 may beembodied according to other and/or additional transceiver technologies.

The processing circuit 26 is depicted in FIG. 2 as including the ADC andDAC. For sensing functionality, the ADC converts the conditioned signalfrom the signal conditioning circuit 24 and digitizes the signal forfurther processing by the microcontroller and/or DSP. The ADC may alsobe used to convert analogs inputs that are received via the inputinterface 42 to a digital format for further processing by themicrocontroller. The ADC may also be used in baseband processing ofsignals received via the communications circuit 40. The DAC convertsdigital information to analog information. Its role for sensingfunctionality may be to control the emission of signals, such as opticalsignals or acoustic signals, from the sensors 22. The DAC may further beused to cause the output of analog signals from the output interface 44.Also, the DAC may be used to convert the digital information and/orinstructions from the microcontroller and/or DSP to analog signals thatare fed to the transmitter circuit. In some embodiments, additionalconversion circuits may be used.

The microcontroller and the DSP provide processing functionality for thewearable device 12. In some embodiments, functionality of bothprocessors may be combined into a single processor, or furtherdistributed among additional processors. The DSP provides forspecialized digital signal processing, and enables an offloading ofprocessing load from the microcontroller. The DSP may be embodied inspecialized integrated circuit(s) or as field programmable gate arrays(FPGAs). In one embodiment, the DSP comprises a pipelined architecture,which comprises a central processing unit (CPU), plural circular buffersand separate program and data memories according to a Harvardarchitecture. The DSP further comprises dual busses, enabling concurrentinstruction and data fetches. The DSP may also comprise an instructioncache and I/O controller, such as those found in Analog Devices SHARC®DSPs, though other manufacturers of DSPs may be used (e.g., Freescalemulti-core MSC81xx family, Texas Instruments C6000 series, etc.). TheDSP is generally utilized for math manipulations using registers andmath components that may include a multiplier, arithmetic logic unit(ALU, which performs addition, subtraction, absolute value, logicaloperations, conversion between fixed and floating point units, etc.),and a barrel shifter. The ability of the DSP to implement fastmultiply-accumulates (MACs) enables efficient execution of Fast FourierTransforms (FFTs) and Finite Impulse Response (FIR) filtering. Some orall of the DSP functions may be performed by the microcontroller. TheDSP generally serves an encoding and decoding function in the wearabledevice 12. For instance, encoding functionality may involve encodingcommands or data corresponding to transfer of information to theelectronics device 14 (or a device of the computing system 20 in someembodiments). Also, decoding functionality may involve decoding theinformation received from the sensors 22 (e.g., after processing by theADC).

The microcontroller comprises a hardware device for executingsoftware/firmware, particularly that stored in memory 28. Themicrocontroller can be any custom made or commercially availableprocessor, a central processing unit (CPU), a semiconductor basedmicroprocessor (in the form of a microchip or chip set), amacroprocessor, or generally any device for executing softwareinstructions. Examples of suitable commercially availablemicroprocessors include Intel's® Itanium® and Atom® microprocessors, toname a few non-limiting examples. The microcontroller provides formanagement and control of the wearable device 12, including determiningphysiological parameters or location coordinates or other contextualdata based on the sensors 22, and for enabling communication with theelectronics device 14 (and/or a device of the computing system 20 insome embodiments).

The memory 28 can include any one or a combination of volatile memoryelements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM,etc.)) and nonvolatile memory elements (e.g., ROM, Flash, solid state,EPROM, EEPROM, etc.). Moreover, the memory 28 may incorporateelectronic, magnetic, and/or other types of storage media.

The software in memory 28 may include one or more separate programs,each of which comprises an ordered listing of executable instructionsfor implementing logical functions. In the example of FIG. 2, thesoftware in the memory 28 includes a suitable operating system and theapplication software 30, which in one embodiment, runs a health andwellness program (though programs for other services, includingbusiness, training, etc. may be used) that includes a plurality ofsoftware modules 32-38 for implementing certain embodiments of aproximity triggered sampling system based on the output from the sensors22 and optionally other inputted data. The raw data from the sensors 22may be used by algorithms of the sensor measurement module 32 todetermine various environmental, physiological and/or behavioralmeasures (e.g., heart rate, biomechanics, such as swinging of the arms),and may also be used to derive other parameters, such as energyexpenditure, heart rate recovery, calories, aerobic capacity (e.g., VO2max, etc.), among other derived measures of physical performance. Insome embodiments, these derived parameters may be computed externally(e.g., at the electronics devices 14 or one or more devices of thecomputing system 20) in lieu of, or in addition to, the computationsperformed local to the wearable device 12.

The operating system essentially controls the execution of computerprograms, such as the application software 30 and associated modules32-38, and provides scheduling, input-output control, file and datamanagement, memory management, and communication control and relatedservices. The memory 28 may also include user data, including weight,height, age, gender, goals, body mass index (BMI) that are used by themicrocontroller executing the executable code of the algorithms toaccurately interpret the measured proximity data, physiological,psychological, and/or behavioral data. The user data may also includehistorical data relating past recorded data to prior contexts. In someembodiments, user data may be stored elsewhere (e.g., at the electronicsdevice 14 and/or a device of the remote computing system 20). The memorymay also store proximity graphs generated repeatedly by the predictionmodule 34 for analysis.

The software in memory 28 comprises a source program, executable program(object code), script, or any other entity comprising a set ofinstructions to be performed. When a source program, then the programmay be translated via a compiler, assembler, interpreter, or the like,so as to operate properly in connection with the operating system.Furthermore, the software can be written as (a) an object orientedprogramming language, which has classes of data and methods, or (b) aprocedure programming language, which has routines, subroutines, and/orfunctions, for example but not limited to, C, C++, Python, Java, amongothers. The software may be embodied in a computer program product,which may be a non-transitory computer readable medium or other medium.

The input interface(s) 42 comprises one or more interfaces (e.g.,including a user interface) for entry of user input, such as a button ormicrophone or sensor (e.g., to detect user input) or touch-type displayscreen. In some embodiments, the input interface 42 may serve as acommunications port for downloaded information to the wearable device 12(such as via a wired connection). The output interface(s) 44 comprisesone or more interfaces for the presentation or transfer of data,including a user interface (e.g., display screen presenting a graphicaluser interface, virtual or augmented reality interface, etc.) orcommunications interface for the transfer (e.g., wired) of informationstored in the memory, or to enable one or more feedback devices, such aslighting devices (e.g., LEDs), audio devices (e.g., tone generator andspeaker), and/or tactile feedback devices (e.g., vibratory motor) and/orelectrical feedback devices. For instance, the output interface 44 maybe used to present the electronic messages to the user in someembodiments. In some embodiments, at least some of the functionality ofthe input and output interfaces 42 and 44, respectively, may becombined, including being embodied at least in part as a touch-typedisplay screen for the entry of input and/or presentation of messages,among other data.

Referring now to FIG. 3, shown is an example electronics device 14 inwhich all or a portion of the functionality of a proximity triggeredsampling system may be implemented. In the depicted example, theelectronics device 14 is embodied as a smartphone (hereinafter, referredto as smartphone 14), though in some embodiments, other types of devicesmay be used, including a workstation, laptop, notebook, tablet, home orauto appliance, etc. It should be appreciated by one having ordinaryskill in the art that the logical block diagram depicted in FIG. 3 anddescribed below is one example, and that other designs may be used insome embodiments. The application software 30A, which may includefunctionality to run a health and wellness (e.g., fitness) program,among other programs, comprises a plurality of software modules (e.g.,executable code/instructions) including the sensor measurement module32A, prediction module 34A, interface module 36A, and communicationsmodule (CM) 38A, all of which are similar to like-named modules for theapplication software 30 of the wearable device 12, and omitted here forbrevity except where noted below. In some embodiments, the applicationsoftware 30A may include additional software modules or fewer softwaremodules/functionality. For instance, as described above, in someembodiments, the application software 30A may comprise all of thefunctionality of the proximity triggered sampling system (e.g.,functionality for sensor measurement, prediction of a moment in timewhen a user state is likely to change, and the change in messagefunction characteristics and/or data collection functioncharacteristics), or a subset thereof in some embodiments. Thesmartphone 14 comprises at least two different processors, including abaseband processor (BBP) 46 and an application processor (APP) 48. As isknown, the baseband processor 46 primarily handles basebandcommunication-related tasks and the application processor 48 generallyhandles inputs and outputs and all applications other than thosedirectly related to baseband processing. The baseband processor 46comprises a dedicated processor for deploying functionality associatedwith a protocol stack (PROT STK), such as a GSM (Global System forMobile communications) protocol stack, among other functions. Theapplication processor 48 comprises a multi-core processor for runningapplications, including all or a portion of the application software 30Aand its corresponding component modules 32A-38A. The baseband processor46 and application processor 48 have respective associated memory (e.g.,MEM) 50, 52, including random access memory (RAM), Flash memory, etc.,and peripherals, and a running clock. Note that, though depicted asresiding in memory 52, all or a portion of the modules 32A-38A of theapplication software 30A may be stored in memory 50, distributed amongmemory 50, 52, or reside in other memory.

More particularly, the baseband processor 46 may deploy functionality ofthe protocol stack to enable the smartphone 14 to access one or aplurality of wireless network technologies, including WCDMA (WidebandCode Division Multiple Access), CDMA (Code Division Multiple Access),EDGE (Enhanced Data Rates for GSM Evolution), GPRS (General Packet RadioService), Zigbee (e.g., based on IEEE 802.15.4), Bluetooth, Wi-Fi(Wireless Fidelity, such as based on IEEE 802.11), and/or LTE (Long TermEvolution), among variations thereof and/or other telecommunicationprotocols, standards, and/or specifications. The baseband processor 46manages radio communications and control functions, including signalmodulation, radio frequency shifting, and encoding. The basebandprocessor 46 comprises, or may be coupled to, a radio (e.g., RF frontend) 54 and/or a GSM modem, and analog and digital baseband circuitry(ABB, DBB, respectively in FIG. 3). The radio 54 comprises one or moreantennas, a transceiver, and a power amplifier to enable the receivingand transmitting of signals of a plurality of different frequencies,enabling access to the cellular network 16 (FIG. 1). In embodimentswhere functionality of the proximity triggered sampling system isdistributed among the electronics device 14 and the computing system 20(and optionally the wearable device 12), the radio 54 enables thecommunication of proximity data (from its integrated beacon technologyor other proximity detection functionality) and any other data (acquiredvia sensing functionality of the electronics device 14 and or relayedfrom inputs from a wearable device 12), and the receipt of messages(e.g., from the computing system 20) and/or instructions to activate orincrease data collection functionality (at the electronics device 14 orfor relaying to the wearable device 12). The analog baseband circuitryis coupled to the radio 54 and provides an interface between the analogand digital domains of the GSM modem. The analog baseband circuitrycomprises circuitry including an analog-to-digital converter (ADC) anddigital-to-analog converter (DAC), as well as control and powermanagement/distribution components and an audio codec to process analogand/or digital signals received indirectly via the application processor48 or directly from a smartphone user interface (UI) 56 (e.g.,microphone, earpiece, ring tone, vibrator circuits, touch-screen, etc.).The microphone may be one mechanism for detecting interactions with auser (similar with any microphone functionality of the wearable device).The ADC digitizes any analog signals for processing by the digitalbaseband circuitry. The digital baseband circuitry deploys thefunctionality of one or more levels of the GSM protocol stack (e.g.,Layer 1, Layer 2, etc.), and comprises a microcontroller (e.g.,microcontroller unit or MCU, also referred to herein as a processor) anda digital signal processor (DSP, also referred to herein as a processor)that communicate over a shared memory interface (the memory comprisingdata and control information and parameters that instruct the actions tobe taken on the data processed by the application processor 48). The MCUmay be embodied as a RISC (reduced instruction set computer) machinethat runs a real-time operating system (RTIOS), with cores having aplurality of peripherals (e.g., circuitry packaged as integratedcircuits) such as RTC (real-time clock), SPI (serial peripheralinterface), I2C (inter-integrated circuit), UARTs (UniversalAsynchronous Receiver/Transmitter), devices based on IrDA (Infrared DataAssociation), SD/MMC (Secure Digital/Multimedia Cards) card controller,keypad scan controller, and USB devices, GPRS crypto module, TDMA (TimeDivision Multiple Access), smart card reader interface (e.g., for theone or more SIM (Subscriber Identity Module) cards), timers, and amongothers. For receive-side functionality, the MCU instructs the DSP toreceive, for instance, in-phase/quadrature (I/Q) samples from the analogbaseband circuitry and perform detection, demodulation, and decodingwith reporting back to the MCU. For transmit-side functionality, the MCUpresents transmittable data and auxiliary information to the DSP, whichencodes the data and provides to the analog baseband circuitry (e.g.,converted to analog signals by the DAC).

The application processor 48 operates under control of an operatingsystem (OS) that enables the implementation of a plurality of userapplications, including the application software 30A (e.g., a health andwellness program, though others may be used). The application processor48 may be embodied as a System on a Chip (SOC), and supports a pluralityof multimedia related features including web browsing functionality ofthe communications module 38A to access one or more computing devices ofthe computing system 20 (FIG. 4) that are coupled to the Internet. Forinstance, the application processor 48 may execute the communicationsmodule 38A (e.g., middleware, such as a browser with or operable inassociation with one or more application program interfaces (APIs)) toenable access to a cloud computing framework or other networks toprovide remote data access/storage/processing, and through cooperationwith an embedded operating system, access to calendars, locationservices, reminders, etc. For instance, in some embodiments, theproximity triggered sampling system may operate using cloud computing,where the processing of data received (indirectly via the smartphone 14or directly) from the wearable device 12 and context data (e.g.,location data, environmental data, etc.) and user inputted data andother data received from the smartphone 14, including proximity data,motion sense, accelerations, speed of travel, imaging, radio taginformation (e.g., RFID), etc.), may be achieved by one or more devicesof the computing system 20, and messaging and/or data collectionfunctionality may be implemented at the electronics device 14 (and/orthe wearable device 12). The application processor 48 generallycomprises a processor core (Advanced RISC Machine or ARM), and furthercomprises or may be coupled to multimedia modules (for decoding/encodingpictures, video, and/or audio), a graphics processing unit (GPU),communications interface (COMM) 58, and device interfaces. In oneembodiment, the communications interfaces 58 may include wirelessinterfaces, including a Bluetooth (BT) (and/or Zigbee in someembodiments) module that enable wireless communication with anelectronics device, including the wearable device 12, other electronicsdevices, and a Wi-Fi module for interfacing with a local 802.11 network,according to corresponding software in the communications module 38A.The communications interface 58 may further comprise beacon technologyto acquire proximity data. The application processor 48 furthercomprises, or in the depicted embodiment, is coupled to, a globalnavigation satellite systems (GNSS) transceiver or receiver (GNSS) 60for enabling access to a satellite network to, for instance, providecoordinate location services. In some embodiments, the GNSS receiver 60,in association with GNSS functionality (e.g., the sensor measurementmodule 32A) in the application software 30A, collects contextual data(time and location data, including location coordinates and altitude),and provides a time stamp to the information provided internally or to adevice or devices of the computing system 20 in some embodiments. Notethat, though described as a GNSS receiver 60, other indoor/outdoorpositioning systems may be used, including those based on triangulationof cellular network signals and/or Wi-Fi. In some embodiments, thesensor measurement module 32A of the application software 30A maycompute speed of movement of the smartphone 14 (and/or other sensordata, including acceleration data) for provision of the contextualinformation (e.g., meta information) internally or to the remotecomputing system 20. For instance, the application software 30A may alsocollect information about the means of ambulation, where the GNSS data(which may include time coordinates) may be used by the applicationsoftware 30A to determine speed of travel, which may indicate whetherthe user is moving within a vehicle, on a bicycle, or walking orrunning. In some embodiments, other and/or additional data may be usedto assess the type of activity, including environmental data (e.g.,humidity, pollution, temperature, etc.), physiological data (e.g., heartrate, respiration rate, galvanic skin response, etc.) and/or behavioraldata, enabling for instance, a determination of the emotional and/orpsychological state of the user by the prediction module 34A.

The device interfaces coupled to the application processor 48 mayinclude the user interface 56, including a display screen. The displayscreen, in some embodiments similar to a display screen of the wearabledevice user interface, may be embodied in one of several availabletechnologies, including LCD or Liquid Crystal Display (or variantsthereof, such as Thin Film Transistor (TFT) LCD, In Plane Switching(IPS) LCD)), light-emitting diode (LED)-based technology, such asorganic LED (OLED), Active-Matrix OLED (AMOLED), retina or haptic-basedtechnology, or virtual/augmented reality technology. For instance, theinterface module 36A may use the display screen to present web pages,personalized electronic messages, and/or other documents or datareceived from the computing system 20 and/or the display screen may beused to present information (e.g., personalized electronic messages) ingraphical user interfaces (GUIs) rendered locally in association withthe interface module 36A. Other user interfaces 56 may include a keypad,microphone, speaker, ear piece connector, I/O interfaces (e.g., USB(Universal Serial Bus)), SD/MMC card, among other peripherals. Alsocoupled to the application processor 48 is an image capture device(IMAGE CAPTURE) 62. The image capture device 62 comprises an opticalsensor (e.g., a charged coupled device (CCD) or a complementarymetal-oxide semiconductor (CMOS) optical sensor). The image capturedevice 62 may be used to detect various physiological parameters of auser, including blood pressure based on remote photoplethysmography(PPG). Also included is a power management device 64 that controls andmanages operations of a battery 66. The components described aboveand/or depicted in FIG. 3 share data over one or more busses, and in thedepicted example, via data bus 68. It should be appreciated by onehaving ordinary skill in the art, in the context of the presentdisclosure, that variations to the above may be deployed in someembodiments to achieve similar functionality.

In the depicted embodiment, the application processor 48 runs theapplication software 30A, which in one embodiment, includes a pluralityof software modules (e.g., executable code/instructions) including thesensor measurement module (SMM) 32A, a prediction module (PM) 34A, aninterface module (IM) 36A, and a communications module (CM) 38A. In someembodiments, GNSS (or like) functionality may be incorporated in thecommunication module 38A or in a separate position determining module,wherein the GNSS functionality operates with the GNSS receiver 60 tointerpret the data to provide a location and time of the user activity.The GNSS functionality of the application software 30A provides locationcoordinates (and a corresponding time) of the user based on the GNSSreceiver input. In some embodiments, the GNSS functionality of theapplication software 30A cooperates with local or external locationservicing services, wherein the GNSS functionality of the applicationsoftware 30A receives descriptive information and converts theinformation to latitude and longitude coordinates. In one embodiment,the communication module 38A, in conjunction with the communicationsinterface 58, enables the receipt from, and/or communication of data to,the wearable device 12 (FIG. 2), and also the transmission of beacons toone or more other devices, and the receipt of beacons from one or moreother devices. From the receipt of beacons from other devices, theapplication software 30A (e.g., the prediction module 34A) can determineproximity graphs and determine a likelihood of a user state change (andhence meaningful moments) to communicate a message to the user and/oractivate (or increase) data collection functionality characteristics asexplained further below. The communications module 38A may enableoperations according to any one or more of a variety of technologies,including BT, NFC, RFID, etc. The communications module 38A furtherincludes network interfacing software, including browser software, toaccess the Internet (and in particular, one or more devices of thecomputing system 20). The interface module 36A may render a GUI on thedisplay screen (e.g., user interface 56) for the presentation ofmessages based on the predicted meaningful moment (user state change) bythe prediction module 34A, and/or present a message audibly (e.g.,according to speaker functionality of the electronics device 14). Theinterface module 36A, through presentation of the messages, may alsoenable via the GUI input by the user in response to questions presentedin the message. In some embodiments, one or more of the software modulesand/or corresponding functionality of the application software 30A maybe further distributed among additional software modules, or in someembodiments, performed at other devices in addition to, or in lieu of,implementation at the smartphone 14. The sensor measurement module 32Acomprises executable code to process the signals (and associated data)measured by the sensors (of the wearable device 12 as communicated tothe smartphone 14, or based on sensors integrated within the smartphone14) and record and/or derive physiological parameters, such as heartrate, blood pressure, respiration, perspiration, etc. The sensormeasurement module 32A may activate a data collection function in theelectronics device 14, including heart rate monitoring (e.g., via theimage capture device 62), location tracking (via the GNSSfunctionality), audio recording functionality, motion sensefunctionality (e.g., accelerometer to sense appendage or torsomovement), environmental parameter/condition monitoring, at the momentin time predicted by the prediction module 34A, or increase sampling ofdata from these devices. Note that all or a portion of theaforementioned hardware and/or software of the smartphone 14 may also bereferred to herein as a processing circuit in some embodiments.

In some embodiments, all of the functionality of the proximity triggeredsampling system may be implemented at the electronics device 14, and insome embodiments, functionality of the proximity triggered samplingsystem may be implemented among any combination of the electronicsdevice 14, the wearable device 12, and the computing system 20. Forinstance, data collection may be achieved at the electronics device 14(e.g., via image capture recording of heart rate, location tracking,beacon reception, etc.) and/or received at the electronics device 14from wireless communications from the wearable device 12, andcommunicated to the computing system 20 for prediction processing.Messages and/or instructions for activation and/or increase/decrease ofdata collection functionality (e.g., data sampling) may be communicatedfrom the computing system 20 to the electronics device 14 forimplementation at the electronics device 14. In some embodiments,prediction processing and activation of messaging and/or data collectionfunctionality may be performed entirely internal to the electronicsdevice 14. In some embodiments, messages may be accessed from a remotedata storage of the computing system 20 based on the prediction of anappropriate time to send messages. These and/or other variations may beimplemented, and hence are contemplated to be within the scope of thedisclosure.

Referring now to FIG. 4, shown is a computing device 70 that maycomprise a device or devices of the remote computing system 20 (FIG. 1)and which may comprise all or a portion of the functionality of aproximity triggered sampling system. Functionality of the computingdevice 70 may be implemented within a single computing device as shownhere, or in some embodiments, may be implemented among plural devices(i.e., that collectively perform the functionality described below). Inone embodiment, the computing device 70 may be embodied as anapplication server device, a computer, among other computing devices.One having ordinary skill in the art should appreciate in the context ofthe present disclosure that the example computing device 70 is merelyillustrative of one embodiment, and that some embodiments of computingdevices may comprise fewer or additional components, and/or some of thefunctionality associated with the various components depicted in FIG. 4may be combined, or further distributed among additional modules orcomputing devices in some embodiments. The computing device 70 isdepicted in this example as a computer system, including a computersystem providing functionality of an application server. It should beappreciated that certain well-known components of computer systems areomitted here to avoid obfuscating relevant features of the computingdevice 70. In one embodiment, the computing device 70 comprises aprocessing circuit 72 comprising hardware and software components. Insome embodiments, the processing circuit 72 may comprise additionalcomponents or fewer components. For instance, memory may be separatefrom the processing circuit 72. The processing circuit 72 comprises oneor more processors, such as processor 74 (PROCES), input/output (I/O)interface(s) 76 (I/O), and memory 78 (MEM), all coupled to one or moredata busses, such as data bus 80 (DBUS). The memory 78 may include anyone or a combination of volatile memory elements (e.g., random-accessmemory RAM, such as DRAM, and SRAM, etc.) and nonvolatile memoryelements (e.g., ROM, Flash, solid state, EPROM, EEPROM, hard drive,tape, CDROM, etc.). The memory 78 may store a native operating system(OS), one or more native applications, emulation systems, or emulatedapplications for any of a variety of operating systems and/or emulatedhardware platforms, emulated operating systems, etc. In someembodiments, the processing circuit 72 may include, or be coupled to,one or more separate storage devices.

For instance, in the depicted embodiment, the processing circuit 72 iscoupled via the I/O interfaces 76 to user profile data structures (UPDS)82 and a messages data structures (MDS) 84, explained further below. Insome embodiments, the user profile data structures 82 and messages datastructures 84 may be coupled to the processing circuit 72 directly viathe data bus 80 (e.g., stored in a storage device (STOR DEV)) or coupledto the processing circuit 72 via the I/O interfaces 76 and the network18 via one or more network-connected storage devices. In someembodiments, the user profile data structures 82 and messages datastructures 84 may be stored in a single device or distributed amongplural devices. Though depicted in association with the computing device70, some embodiments of a proximity triggered sampling system may usethe data structures 82 and/or 84 locally to the user devices (e.g., atthe wearable device 12 and/or the electronics device 14). Thoughdescribed as separate data structures, in some embodiments, the contentstored by the user profile data structures 82 and messages datastructures 84 may be combined into a single data structure. The userprofile data structures 82 and messages data structures 84 may be storedin persistent memory (e.g., optical, magnetic, and/or semiconductormemory and associated drives). In some embodiments, the user profiledata structures 82 and messages data structures 84 may be stored inmemory 78.

The user profile data structures 82 are configured to store user profiledata. In one embodiment, the user profile data comprises demographicsand user data, including physiological data and contextual datacorresponding to user monitored activity. Contextual data may becommunicated from the wearable device 12 and/or the electronics device14, or from other devices. For instance, some user contextual data maybe collected from existing personal resources, including from socialmedia websites or from a blog or from a patient sharing group, calendarsof the user or of people with whom the user interacts, among othersources. The user profile data structures 82 may be accessed by theprocessor 74 executing software in memory 78 to supplement proximitydata received from the wearable device 12, electronics device 14, and/orother devices, enabling a prediction of user state changes andultimately a moment in time to change messaging function characteristicsand/or data collection function characteristics at the computing system20 (or communicating the message or instructing the same functionalityat the wearable device 12 and/or electronics device 14).

The messages data structure 84 comprises predefined messages (e.g., oneway messages, two-way messages) to be communicated to the user devices12 and/or 14, and in some embodiments, may include a template or beupdated manually or via a machine learning algorithm (e.g., regression,random forest, (deep) neural networks, etc.) to personalize themessages. In some embodiments, one or more of the content stored in theuser profile data structures 82 may also be stored as meta informationin the messages data structures 84.

The user profile data structure 82 may also include current orcontemporaneous activity data for the user that is communicated to thecomputing device 70 during synch operations with the smartphone 14and/or wearable device 12 or as communicated from a third party serverdevice (e.g., medical facility, fitness tracking service, etc.).Additional data structures may be used to record similar information forother users. The user profile data structures 82 may be updatedperiodically, aperiodically, and/or in response to one or more events.

In the embodiment depicted in FIG. 4, the memory 78 comprises anoperating system (OS) and application software (ASW) 30B, the latterwhich may execute a health and wellness program, among other programs insome embodiments. The application software 30B comprises a plurality ofsoftware modules (e.g., executable code/instructions) including thesensor measurement module (SMM) 32B, a prediction module (PM) 34B, aninterface module (IM) 36B, and a communications module (CM) 38B. In someembodiments, the application software 30A may include additional orfewer software modules and/or functionality. The sensor measurementmodule 32B analyzes measurements and/or user-inputted profile data fromthe wearable device 12 and/or smartphone 14 (e.g., accessed from theuser profile data structure 82) as part of the health and wellnessprogram. For instance, the sensor measurement module 32B receives andanalyzes the activity data, environmental data, physiological data,psychological data, location data, demographics, etc. of orcorresponding to the user and provides a dashboard of information thatmay include progress towards a given goal (e.g., fitness goals),comparisons to the general population (or user-configured population ofpeople), and/or performance data for various activities (e.g.,physiological measurements, derived measurements including caloric loss,energy expenditure, etc.). The sensor measurement module 32B may alsoprovide feedback to the prediction module 34 as to the physiologicalstate of the user based on the physiological measures. Thecommunications module 38B receives the activity data/user profile dataand proximity data and other contextual data from the wearable device 12and/or electronics device 14, and also communicates electronic messagesand/or instructions to the wearable device 12 and/or the electronicsdevice 14. The communications module 38B generally enablescommunications among network-connected devices and provides web and/orcloud services, among other software such as via one or more APIs. Insome embodiments, the communications module 38B may be separate from theapplication software 30B. In one example operation, the communicationsmodule 38B may receive (via I/O interfaces 76) input data from thewearable device 12 and/or the electronics device 14 that includes senseddata (e.g., proximity data, physiological data, environmental data,location data, etc.), context data, user-inputted data, data fromthird-party databases (e.g., medical data base, health program providerdata, etc.), data from social media (e.g. from other computing devices),data from external devices (e.g., weight scales, optionallyenvironmental sensors, etc.), among other data. The input may becontinual, intermittent, and/or scheduled. The communications module 38Bmay, in one embodiment, be activated by the application software 30Bbased on a determined meaningful moment (by the prediction module 34B)to communicate electronic messages to the wearable device 12 and/orelectronics device 14 that are informational and do not require orsolicit a user reply (e.g., one-way messages) or that solicit user input(e.g., two-way messages). In some embodiments, the communications module38B may cooperate with the interface module 36B to provide web servicesfor the wearable device 12 and/or electronics device 14, enabling theaccess and download of web pages to enable the user, using browserfunctionality in the wearable device 12 and/or electronics device 14, toreceive messages and optionally reply to messages. In general, thecommunications module 38B may comprise a web service component or cloudcomponent which is accessed by a client application (e.g., browser,etc.) residing at the wearable device 12 and/or electronics device 14.

Execution of the application software 30B (including software modules32B-38B) may be implemented by the processor 74 under the managementand/or control of the operating system. The processor 74 may be embodiedas a custom-made or commercially available processor, a centralprocessing unit (CPU) or an auxiliary processor among severalprocessors, a semiconductor based microprocessor (in the form of amicrochip), a macroprocessor, one or more application specificintegrated circuits (ASICs), a plurality of suitably configured digitallogic gates, and/or other well-known electrical configurationscomprising discrete elements both individually and in variouscombinations to coordinate the overall operation of the computing device70.

The I/O interfaces 76 comprise hardware and/or software to provide oneor more interfaces to the Internet 18, as well as to other devices suchas a user interface (UI) (e.g., keyboard, mouse, microphone, displayscreen, etc.) and/or the data structures 82-84. The user interfaces mayinclude a keyboard, mouse, microphone, immersive head set, displayscreen, etc., which enable input and/or output by an administrator orother user. The I/O interfaces 76 may comprise any number of interfacesfor the input and output of signals (e.g., analog or digital data) forconveyance of information (e.g., data) over various networks andaccording to various protocols and/or standards. The user interface (UI)is configured to provide an interface between an administrator orcontent author (e.g., for the generation of messages or messagetemplates) and the computing device 70. In one embodiment, the userinterface enables the administrator to craft the message templates, andonce defined, the messages may be generated automatically (and updatedvia learning algorithms). The administrator may input a request via theuser interface, for instance, to manage the user profile data structures82 and/or the messages data structures 84. Updates to the datastructures 82 and/or 84 may also be achieved without administratorintervention.

When certain embodiments of the computing device 70 are implemented atleast in part with software (including firmware), as depicted in FIG. 4,it should be noted that the software (e.g., including the applicationsoftware 30B (and associated modules 32B-38B) can be stored on a varietyof non-transitory computer-readable medium for use by, or in connectionwith, a variety of computer-related systems or methods. In the contextof this document, a computer-readable medium may comprise an electronic,magnetic, optical, or other physical device or apparatus that maycontain or store a computer program (e.g., executable code orinstructions) for use by or in connection with a computer-related systemor method. The software may be embedded in a variety ofcomputer-readable mediums for use by, or in connection with, aninstruction execution system, apparatus, or device, such as acomputer-based system, processor-containing system, or other system thatcan fetch the instructions from the instruction execution system,apparatus, or device and execute the instructions.

When certain embodiments of the computing device 70 are implemented atleast in part with hardware, such functionality may be implemented withany or a combination of the following technologies, which are allwell-known in the art: a discrete logic circuit(s) having logic gatesfor implementing logic functions upon data signals, an applicationspecific integrated circuit (ASIC) having appropriate combinationallogic gates, a programmable gate array(s) (PGA), a field programmablegate array (FPGA), relays, contactors, etc.

Referring now to FIG. 5, shown is a schematic diagram 86 thatillustrates example changes in proximity information as a function oftime according to an embodiment of a proximity triggered samplingsystem. The diagram 86 shows a timeline (on the horizontal axis, t=0through t=T5) of social interactions between users as detected viabeacon communications (or other proximity detection technology,including Wi-Fi) among user device D0 and other user devices D1 and D2.The timeline of social interactions help illustrate an example where ameaningful sampling moment corresponding to a change in state of theuser (of D0) is predicted (e.g., by the prediction module 34, 34A, or34B of FIGS. 2-4). Digressing briefly, the proximity triggered samplingsystem operates under several premises, namely that interactions withother people influence a user's emotional state, and the emotional statechanges originating from person-to-person interactions depend on thenumber of people involved in the interaction, the duration ofinteraction, and potentially other features, such as location ofinteraction, and people involved. Since it is highly likely that userstate (e.g., emotional) will change as a result of an interaction, it ismore likely to get a representative description of the user state, bysampling at these moments, instead of at random times. The premise ofuser state changes as a result of interacting with others has beensupported by experimental data of the assignee of the presentapplication, and other research. For instance, experimental data hasshown that users' heart rate and emotions changes as a result of socialinteractions with others. The findings also suggest that the number ofpeople involved in the interaction can also play a role in how userstate changes. Interaction characteristics can be unobtrusivelymonitored using proximity sensors (e.g., beacon technology, such asiBeacon®) in the user devices D0-D3. Certain embodiments of a proximitytriggered sampling system monitors the proximity characteristics, todetect the change points in time, and to push to a user a timely message(e.g., one-way, two-way) and/or implement a change in data collectionfunction characteristics at these moments. In FIG. 5, the applicationsoftware 30 of the D0 detects when person-to-person interactions havecommenced and ended (e.g., the number of devices in the proximity ofuser changes from greater than 1 to 0) and uses these moments to cause achange in messaging function characteristics (e.g., push an emotionalstate question) and/or data collection function characteristics. Morespecifically, in FIG. 5, it is observed that at time t=0, the observeduser (user 0, using D0) is alone. At t=T1 the user is having a socialcontact with one person, and at time T2, with two people (e.g.,corresponding to devices D2 and D1). At T3 the interaction is only withone person again. The interaction ends at T4, and the intervention(sampling) is delivered (or initiated/activated) at T5.

Referring now to FIG. 6, shown is a flow diagram that illustrates anexample proximity triggered sampling process 88. It should beappreciated by one having ordinary skill in the art, in the context ofthe present disclosure, that the process 88 depicted in FIG. 6 is butone example, and that in some embodiments, the process 88 may haveadditional or fewer steps. The process 88 assumes prediction processing(e.g., by the application software 30B executing in the computing system20, and in particular, the computing device 70 (FIG. 4), such as via acloud platform implementation), with the computing device 70 receivingdata from, and communicating messages and/or instructions to, userdevice D0 (e.g., a wearable device 12 or electronics device 14, FIG. 1).It should be appreciated by one having ordinary skill in the art thatthe depicted example is illustrative of one embodiment, and as indicatedabove, the prediction processing may be performed at one or more of theuser devices in some embodiments. As illustrated in FIG. 6, the userdevice (D0) detects other user devices, each of the user devicesequipped with proximity sensing functionality (e.g., iBeacon devices,Wi-Fi, etc.). Using beacon signal strength information, the applicationsoftware 30B creates proximity graphs (90). Changes in the proximitygraphs are monitored in real time by the application software 30B, andproximity features are calculated (92) by the application software 30B(e.g., prediction module 34B). Based on the proximity features (and insome embodiments, user data), sampling method characteristics (e.g.,content-related and/or non-content related message features, includingEMA (e.g., message) type, content, time, such as determining when to askthe user a question based on changes in the proximity graph may be basedon changes in ego density, etc.) are determined (94), which are thenused to select the appropriate sampling content (96), which is thendelivered as a change in message function characteristics and/or datacollection function characteristics (e.g., deliver a message,instructions to start a measurement, etc.) to the user's device D0. Notethat in some embodiments, selection (96) may be replaced with, orsupplemented by, generation. For instance, templates (or rules ornatural language processing algorithms) may be used and the content tobe communicated to the user may be automatically generated as describedpreviously.

The user response is tracked and stored in a database (e.g., userprofile data structure 82, FIG. 4), together with the proximityfeatures. Using the (historical) information in the database, samplingfeature calculation and selection can be improved (for examplepersonalized according to the user, devices in the proximity graph,location, etc.) by applying data mining techniques. For instance,learning algorithms may be used to determine the best way to communicatewith the user. If, for certain combinations of the proximity featuresand sampling content features, it is observed that the user is notresponsive, then this information can be taken into account in futurecalculations. For example, if the proximity triggered sampling systemlearns that after a 1-1 meeting that happens regularly on Friday, theuser is always responding, this can result in increasing the number ofquestions that are asked after this meeting. In some embodiments, theproximity triggered sampling system uses multiple data sources/features,which include proximity features, sampling content features (alsoincluding time, environment, lexicon, semantics, sentiments, etc.), userstate features, for example if the user is tracking HR, movements,speech, temperature, etc., and user input features (e.g., how the userhas responded in the past). The aforementioned sources/features areprocessed together to find/predict what is the best combination of thesesources/features. In one embodiment, based on the three or more of thestored user response, the user state, features correspond to sampling,or the proximity features, the system enhances future proximity featurecomputations and the features corresponding to the sampling.

Note that the description of the process 88 above describes thedetermination of the sampling method characteristics based on proximitydata, and in some embodiments, proximity data and user data (userfeatures). Certain embodiments of a proximity triggered sampling systempredict potential points in time to sample for the user's subjectiveinput. To determine these sampling moments and sampling characteristics,in one embodiment, all available data may be used, including (objective)proximity real-time characteristics, (objective) real time user datacharacteristics (e.g., heart rate, etc.), past (subjective) user inputdata characteristics, and past (objective) proximity and user datacharacteristics.

With continued reference to FIGS. 5-6, attention is directed to FIG. 7,which schematically illustrates an example proximity graph 98. Certainembodiments of a proximity triggered sampling system (e.g., theprediction module 34B of the application software 30B) make use of theproximity graphs 98 to monitor the proximity of users in real time. Theproximity graph 98 correspond to a proximity of one or more otherdevices to a user device and to each other. Changes in the proximityinformation determined from the proximity graphs 98 are detected in realtime, and this information (among other information in some embodiments)is used as a trigger to change (e.g., activate, increase, decrease)messaging function characteristics and/or a data collection functioncharacteristics (e.g., to deliver the EMAs, or in general, messagesand/or to sample or collect user data). The proximity graphs 98 comprisea computational structure that represents the proximity of devices(e.g., wearable devices 12 and/or electronic devices 14). Nodes in theproximity graph 98 are devices, and an edge indicates that these devicesare in proximity.

In one embodiment, the prediction module 34B monitors the changes in theproximity graph in real time based on the calculation by the predictionmodule 34B of proximity features, which includes any one or combinationof a density of the proximity graph 98, a number of connections (e.g.,where connections refer to a condition where a user device is receivinga strength signal (above a certain threshold) and for a specific time(above a certain threshold) that can be interpreted that the user is inproximity/interaction with the owner of that device) of an individual, arate of change of connections, a duration of particular connections, andan identity (e.g., device IDs) of connections, and proximity featurescorresponding to devices of other users in proximity to the user (e.g.,a user may be in contact with user A, and if accessible, the proximityfeatures from user A's device may also be considered a proximity featurefor the current user). In one embodiment, an interaction (e.g., socialinteraction) is defined as (at least) one other device (D1) being inproximity of a user's device (D0) for more than two (2) minutes, andthen (after a specific, predetermined time, say in seconds (e.g., lessthan 60 seconds)) no device being in the proximity of D0, after D1 (ormore devices if that is the case) leaves. A start of the socialinteraction is the moment D0 and D1 get in sufficient proximity of eachother, and an end of the social interaction is when D1 is no longer insufficient proximity of D0. What is deemed a sufficient proximity may bedependent on the number of devices in proximity of D0. For instance,referring to the interpersonal distances of man (defined by Edward T.Hall's work in the field of Proxemics), in the case of a single device,the distance is defined as being less than 1.2 meters; in the case oftwo or three devices, the distance is defined as being less than 2.1meters; and in the case of more than three devices, the distance isdefined as being less than 3.7 meters. Note that other definitions fordetermining the sufficiency of proximity to other devices may be used,depending on the conditions and/or goals or other design constraints ofthe system and/or environment in which the system is used. The distancebetween devices is approximated from the iBeacon signal strength inwhich a greater signal strength indicates a shorter distance. An examplemethod to calculate distances using iBeacon® (a protocol developed byApple®) may be found in technical forums, such as on-line stack overflowforums on iBeacon® distancing, which provides that the distance estimateprovided by iOS is based on the ratio of the iBeacon (e.g., formatted asseveral bytes including a unique identifier recognized by compatibledevices/applications) with signal strength (RSSI) over the calibratedtransmitter power (txPower). The txPower is the known measured signalstrength in RSSI at one (1) meter away. Each iBeacon must be calibratedwith this txPower value to allow accurate distance estimates.

The prediction module 34B triggers assessments (e.g., sampling) based onthe calculated proximity features. In one example, the density of asubgraph within the proximity graph increases quickly, which may implythat nodes within that subgraph have formed an (adhoc) meeting. If aftera certain period of time, for instance twenty (20) minutes, the densityof the subgraph quickly decreases, this means that the meeting hasended. These two observations, the forming of a group and the subsequentdissolving of that group, provide a suitable pattern for sending amessage to members of the group. Using signal strength information fromthe iBeacons, kinesthetic factors related to the distance betweenparticipants can be calculated. The user state change is related to thechange in proximity graphs, which is a result of an observed change inthe monitored signal strengths. In other words, greater user emotionalstate changes can be expected as a result of closer proximityinteractions. In one embodiment, the prediction module 34B detects thestart and end of social interaction, as described above, and delivers amessage (or in general, samples user (sensor) data) shortly (e.g.,within sixty (60) seconds) after the end of the interaction.

In some embodiments, social interaction features can be approximatedfrom the proximity graph features, and these can be used whiledetermining the moment and type (content) of messages. A non-exhaustivelist of examples of interaction features that can be extracted from theproximity graphs and how they can be used for triggering and selectingmessages are shown in the following Table 1.

TABLE 1 Proximity graph feature EMA feature Explanations Logic Meetingduration Delivery time Delivery time of EMA is People need determined asa function of more recovery a meeting duration. E.g. If time after longthe meeting was short meetings (<=30 min) deliver the questionnaire 1min after end of the meeting. If the meeting was long (>30 min), deliverthe questionnaire 5 min after end of the meeting. Meeting durationNumber of Questionnaire length is a More user state questions functionof meeting factors change duration. After short after longer meetingsuse shorter meetings questionnaires (e.g. 2 questions), and after longmeeting use longer questionnaires (e.g. 4 questions). Number of people(Emotion) The amount of details Emotions are in the meeting questionasked from the user is a more influenced specificity and function ofnumber of in small group detail people that were present in meetings themeeting. After meetings with less than 4 people ask more specific ordetailed questions. For example, in addition to asking the users toselect or input their emotions also ask them to rate the strength oftheir emotion. Proximity of people Context and Ask questions related toContext in the meeting location the context and location if determineshow questions proximity of participants is close people different thannormal will stay during meetings Identity of people Person specific Askquestions specific to Who is present in the meeting, or questions thepeople present in the in the meeting type (e.g. meeting can stronglycolleagues) or role influence how (e.g. direct emotions manager) of thechange people in the meeting

Table 1 illustrates an example look-up table that illustrates howmessages may be selected based on assessments of the proximity graphfeatures. Note that the various features presented in Table 1 are onlysome specific example, which do not cover all possible combinations. Ingeneral, certain embodiments of a proximity triggered sampling systemcan model the question parameters as a function of all or a portion ofthe proximity features. One relationship between proximity features andthe message features can be represented using a rule-based look up tableapproach as shown in Table 1, or can be modelled by the predictionmodule 34B as a mathematical function, or in some embodiments, a datamining approach (e.g. using neural networks) can be used.

Although the various embodiments have been described in the context ofhuman-human social interactions, in some embodiments of a proximitytriggered sampling system, human-computer interaction changes may bemonitored in an effort to predict potential sampling methods. Forexample, user engagement in a conversation with a virtualagent/assistant (e.g., chatbot) can be detected, and the message can bedelivered (or in general, a message function characteristic changed)when the interaction is over (and/or a data collection functioncharacteristic may be changed). Such detection may be via detection of aswitch or button selection on the device and/or from detection ofmovement (e.g., gestures) if the device is equipped with movementsensors, such as accelerometers. A signal received from themachine/computer/device that the user is communicating with may be usedin some embodiments to indicate a status of the communications orinteractions (e.g., start, end, continuing, etc.).

Further, though operations have been emphasized using proximity data asinput to the prediction module 34B, in some embodiments, other sensordata may be used, wherein a context of the social interaction can bedetected and the sampling can be adapted accordingly. For example, ifGNSS data (e.g., GPS data) is available, social interactions can bedifferentiated as between-family or between work colleagues. Anotherexample is using accelerometer data, where a user activity may bedetected, including running or exercising in the gym together withsocial partners. Other examples include, without limitation, monitoringof speech (e.g., voice) and/or dialog characteristics, monitoring offacial and/or facial features (e.g., lips, eyes, etc.), monitoring ofgestures (e.g., head, hands, torso, legs, etc.). Depending on thecontext information, the type and content of the messages can beadapted. In an example implementation, different messaging options andtheir relation to the user context can be stored in a look-up table.Further, though proximity data is described as a mechanism for detectinga user interaction for the prediction of user state change, other and/oradditional mechanisms may be used for detection of interactions (withmachine or humans), include voice detection using microphones, meetingsvia an electronic calendar, SKYPE conversations, social media/websiteconversations, etc.

In view of the description above, it should be appreciated that oneembodiment of a computer-implemented, proximity triggered samplingmethod, depicted in FIG. 8 and referred to as a method 100 andencompassed between start and end designations, comprises receiving datacorresponding to an interaction with a user (102); based on the receiveddata, predicting a moment in time when a state of the user is likely tochange (104); and causing a change in one or a combination of messagefunction characteristics or data collection function characteristics atthe moment in time (106). The change may involve activation of messagefunction characteristics or data collection function characteristics, oran increase or decrease in either or both the message functioncharacteristics or data collection function characteristics. Messagefunction characteristics include whether the message function isactivated or not, a frequency of the messaging, a content or type of themessage, one or any combination of length, size, duration of themessage, a delivery time of the message, among other characteristics.Data collection function characteristics include whether the datacollection is activated or not, a start and stop time of the datacollection, a frequency of the data collection, the type of measurementfunction, a delivery time of the data collection function, among othercharacteristics. For instance, an activation of the message functioncharacteristics may involve communicating messages (one-way or two-waymessages) at a meaningful moment in time. An example of an increase of,say, a data collection function characteristic may involve increasedsampling of a heart rate measurement, or tracking (GNSS) function. Also,an example of a decrease of say, a message function characteristic maybe realized in scenario that, since the sampling characteristics otherthan time can also change, a user may be confronted with a decreasednumber of questions depending on the observed proximity and userfeatures. Also, the state of the user may include one or any combinationof the psychological or physiological state of the user. For instance,the psychological state may include the affective or emotional state,states of readiness, curiosity, absorbed, or conscious, to name a few.The physiological state may manifest itself where a user has just left along meeting and is tired, and/or the physiological state may bereflected by readings of the sensors (e.g., heart rate, blood pressure,skin temperature, VO2max, etc.).

Any process descriptions or blocks in flow diagrams should be understoodas representing modules, segments, or portions of code which include oneor more executable instructions for implementing specific logicalfunctions or steps in the process, and alternate implementations areincluded within the scope of the embodiments in which functions may beexecuted out of order from that shown or discussed, includingsubstantially concurrently or in reverse order, depending on thefunctionality involved, as would be understood by those reasonablyskilled in the art of the present disclosure.

Though various embodiments of a proximity triggered sampling system havebeen disclosed in the context of a lifestyle application, it should beappreciated by one having ordinary skill in the art that applicationsfor the proximity triggered sampling system may be extended to otherfields, including healthcare environments (e.g., as an application forpatients and healthcare providers), commercial applications (e.gh.,monitoring the state of shoppers to provide relevant coupons), educationenvironments (e.g., providing monitoring and interaction options forstudents or teachers), and business (e.g., as a tool for monitoringproductiveness and/or user satisfaction). As another example, certainembodiments of a proximity triggered sampling system may be used in thefinancial industry, wherein the proximity triggered sampling system mayenable a user to make an investment when in a proper state of mind.Further, messages may vary depending on various demographics, includinggender. For instance, messages may be delivered that are more suitableto a man than a woman.

As a further example of variations to the above-description of certainembodiments of a proximity triggered sampling system, though textualfeedback (presentation of the messaging) has been described as via thewearable device 12 and/or electronics device 14, feedback messaging maybe presented audibly (in lieu of, or in addition to text) via speakerfunctionality of either device 12, 14 (FIG. 1) and/or visibly (e.g.,video playback of an avatar). In some embodiments, the messaging may beconveyed via a third party. For instance, the messages may be deliveredto a third person (e.g., the user's coach or trainer or mentor) and inturn conveyed verbatim or substantially verbatim by the third person.The message may be delivered, for instance, to an earpiece worn by thethird person and relayed by the third person to the user.

Note that various combinations of the disclosed embodiments may be used,and hence reference to an embodiment or one embodiment is not meant toexclude features from that embodiment from use with features from otherembodiments. In the claims, the word “comprising” does not exclude otherelements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor or other unit may fulfill thefunctions of several items recited in the claims. The mere fact thatcertain measures are recited in mutually different dependent claims doesnot indicate that a combination of these measures cannot be used toadvantage. A computer program may be stored/distributed on a suitablemedium, such as an optical medium or solid-state medium suppliedtogether with or as part of other hardware, but may also be distributedin other forms. Any reference signs in the claims should be notconstrued as limiting the scope.

At least the following is claimed:
 1. A system, comprising: memorycomprising instructions; and one or more processors configured toexecute the instructions to: receive data corresponding to aninteraction with a user; based on the received data, predict a moment intime when a state of the user is likely to change; and cause a change inone or a combination of message function characteristics or datacollection function characteristics at the moment in time.
 2. The systemof claim 1, wherein the state of the user comprises any one or acombination of a psychological state of the user or a physiologicalstate of the user.
 3. The system of claim 1, wherein the one or moreprocessors are further configured to execute the instructions to causethe change by one or any combination of causing an activation or anincrease or decrease in the message function characteristics at themoment in time or causing a message to be communicated to the user atthe moment in time.
 4. The system of claim 3, wherein the messagecomprises either a one-way communication message that does not solicitdirect feedback from the user or a two-way communication message thatsolicits direct feedback from the user, wherein the message iscommunicated locally or remotely.
 5. The system of claim 1, wherein theone or more processors are further configured to execute theinstructions to cause the change by causing an activation or an increaseor decrease in the data collection function characteristics.
 6. Thesystem of claim 1, wherein the data comprises proximity data, andwherein the one or more processors are further configured to execute theinstructions to predict the moment at least by determining whether theuser is alone or within sufficient proximity to one or more other users.7. The system of claim 6, wherein the data further comprises othersensor data, wherein the one or more processors are further configuredto execute the instructions to further predict the moment at least bydetermining how the other sensor data is changing in time anddetermining if the other sensor data differs from a set of data alreadycollected.
 8. The system of claim 6, wherein the one or more processorsare further configured to execute the instructions to determine one ormore characteristics associated with interactions involving the userbased on the proximity data.
 9. The system of claim 8, wherein the oneor more processors are further configured to execute the instructions topredict the moment further based on the monitoring to detect when anumber of devices in sufficient proximity to a user device has changed.10. The system of claim 1, wherein the sensor data comprises arespective strength of beacon signals received in real time, wherein theone or more processors are further configured to execute theinstructions to: repeatedly generate proximity graphs based on thestrength of the beacon signals, the proximity graphs corresponding to aproximity of one or more other devices to a user device and to eachother; compute proximity features based on changes in the proximitygraphs monitored in real time; store the proximity features in a datastorage device; determine sampling method characteristics based on theproximity features; and select or generate one or any combination of amessage or the data collection function based on the determination. 11.The system of claim 10, wherein the one or more processors are furtherconfigured to execute the instructions to: communicate the message atthe moment in time; track a user response to the message; store the userresponse in the data storage device; and based on the three or more ofthe stored user response, the user state, features correspond tosampling, or the proximity features, enhance future proximity featuresand/or their computations and the features corresponding to the samplingand/or their computation.
 12. The system of claim 10, wherein theproximity features comprise any one or a combination of a proximitybetween the user device and the one or more other user devices, adensity of each proximity graph, a number of connections between theuser device and the one or more other devices, a duration of each of theconnections, an identity of each of the connected devices, or proximityfeatures corresponding to one or more devices in proximity of the userdevice, wherein each connection corresponds to the user device receivinga strength signal above a threshold for a time above a threshold amountof time.
 13. The system of claim 12, wherein the one or more processorsare further configured to execute the instructions to associate aninteraction between a user and another user based on a connection of theuser device to another device, wherein a start of the interaction isbased on a sufficient proximity of the user device to the another userdevice for a specific length of time and an end of the interaction isbased on a subsequent termination of sufficient proximity between theuser device and the another user device.
 14. The system of claim 13,wherein the one or more processors are further configured to execute theinstructions to cause the change by causing an activation of the messagefunction characteristics a specific time after an end of theinteraction, which corresponds to the moment in time.
 15. The system ofclaim 1, wherein the one or more processors are part of a cloudcomputing platform or a user device.
 16. The system of claim 1, whereinthe one or more processors are further configured to execute theinstructions to predict the moment in time when the state of the user islikely to change further based on any one or a combination of sensedhuman-device interactions or signals from one or more additionalsensors.
 17. The system of claim 16, wherein the signals from one ormore additional sensors comprises one or any combination of user deviceposition data, environmental data, facial characteristic data, voicedata, or user movement data.
 18. The system of claim 16, wherein the oneor more processors are further configured to execute the instructions tocause activation or adaptation of any one or combination of thecommunication function characteristics or data collection functioncharacteristics when the human-device interaction has ended.
 19. Anon-transitory computer readable medium comprising executable code that,when the code is executed by one or more processors, causes the one ormore processors to: receive data corresponding to an interaction with auser; based on the received data, predict a moment in time when a stateof the user is likely to change; and cause a change in one or acombination of message function characteristics or data collectionfunction characteristics at the moment in time.
 20. Acomputer-implemented method, comprising: receiving data corresponding toan interaction with a user; based on the received data, predict a momentin time when a state of the user is likely to change; and causing achange in one or a combination of message function characteristics ordata collection function characteristics at the moment in time.